Adaptive Subgradient Methods Adaptive Subgradient Methods for Online Learning and Stochastic Optimization∗ John Duchi

Source URL: stanford.edu

• Convex optimization
• Mathematical optimization
• Stochastic optimization
• Machine learning
• Gradient descent
• Subgradient method
• Stochastic gradient descent
• Proximal gradient methods for learning
• Perceptron
• BroydenFletcherGoldfarbShanno algorithm
• Online machine learning
• Loss functions for classification

Energetic Natural Gradient Descent

Source URL: psthomas.com

• Estimation theory
• Statistical theory
• Parametric model
• Loss function
• Fisher information
• Expectationmaximization algorithm
• Divergence
• Likelihood function
• Gradient descent

Theano Tutorial Theano is a software package which allows you to write symbolic code and compile it onto different architectures (in particular, CPU and GPU). It was developed by machine learning researchers at the Univ

Source URL: colinraffel.com

• Artificial neural networks
• Sigmoid function
• Artificial neuron
• Parameter
• Function
• Gradient descent
• Backpropagation
• Multilayer perceptron
• Activation function
• Theano
• Default argument
• Artificial intelligence

Case Study 1: Estimating Click Probabilities Intro Logistic Regression Gradient Descent Machine Learning/Statistics for Big Data

Source URL: courses.cs.washington.edu

Simple Evolutionary Optimization Can Rival Stochastic Gradient Descent in Neural Networks In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCONew York, NY: ACM Nominated for Best Paper Aw

Source URL: eplex.cs.ucf.edu

• Artificial neural networks
• Applied mathematics
• Artificial intelligence
• Machine learning
• Neuroevolution
• Evolutionary algorithm
• Deep learning
• Backpropagation
• Stochastic gradient descent
• HyperNEAT
• Genetic algorithm
• Multilayer perceptron

Supplementary Materials: Accelerated Stochastic Gradient Descent for Minimizing Finite Sums Atsushi Nitanda Tokyo Institute of Technology and NTT DATA Mathematical Systems Inc. Tokyo, Japan

Source URL: jmlr.org

ADVANCES IN OPTIMIZING RECURRENT NETWORKS Yoshua Bengio, Nicolas Boulanger-Lewandowski and Razvan Pascanu arXiv:1212.0901v2 [cs.LG] 14 DecU. Montreal

Source URL: arxiv.org

• Artificial neural networks
• Applied mathematics
• Machine learning
• Cybernetics
• Recurrent neural network
• Deep learning
• Long short-term memory
• Stochastic gradient descent
• Convolutional neural network
• Rectifier
• Backpropagation
• Gradient descent

Topmoumoute online natural gradient algorithm Nicolas Le Roux University of Montreal Pierre-Antoine Manzagol

Source URL: nicolas.le-roux.name

• Mathematics
• Numerical analysis
• Algebra
• Computational statistics
• Stochastic optimization
• Linear filters
• Numerical linear algebra
• Convex optimization
• Stochastic gradient descent
• Gradient descent
• Artificial neural network
• Mathematical optimization

Gradient Descent Objective Function Some differentiable function f : Rn → R. Gradient Descent Start with some x0, i = 0 and learning rate λ repeat

Source URL: alex.smola.org

Weighted SGD for ?𝓅 Regression with Randomized Preconditioning

Source URL: www.stat.berkeley.edu

• Operations research
• Econometrics
• Convex optimization
• Numerical linear algebra
• Estimation theory
• Preconditioner
• Support vector machine
• Linear regression
• Gradient descent
• Regression analysis
• Stochastic gradient descent
• Linear programming